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Background / Context: 

School fees were abolished in Kenya in 2003. Since that time, gross primary enrollment rates 
have risen above 100% (World Bank, 2011). This dramatic increase in enrollment over a short 
period put considerable strain on the government school system, which did not receive funding 
increases commensurate with the enrollment increases. In 1998, the national student-teacher 
ratio was 28 to 1. In 2011, it was 47 to 1 (World Bank, 2011). In addition to handling large 
classes, Kenyan teachers often deal with space and materials shortages that impair their ability to 
teach effectively (Sifuna, 2007; UNESCO, 2005). 

Perhaps unsurprisingly, given the high student-teacher ratios, limited teacher training, and lack 
of sufficient text materials, reading outcomes for students attending Kenyan primary schools are 
generally poor. The results of a series of assessments conducted over the decade since fee 
abolition converge on a common finding: Kenyan children are not meeting the Ministry of 
Education’s benchmarks and on average read far below grade level (Mugo et al., 2011; National 
Assessment Centre, 2010; Onsomu et al., 2005; Piper, 2010; Piper & Mugenda, 2012; Wasanga, 
Ogle, & Wambua, 2010). For example, the 2011 national Uwezo study found that just 57% of 
third-graders could read basic sentences, and only 30% a second-grade-level story (Mugo et al., 
2011 ). 

Purpose / Objective / Research Question / Focus of Study: 

If children do not learn how to read in the first few years of primary school, they will struggle to 
complete the cycle and are at greater risk of dropping out. It is therefore crucial to identify and 
test interventions that have the potential of making a large impact, can be implemented quickly, 
and are affordable to be taken to scale by the Kenyan government. This is the goal of the PRIMR 
Initiative — to test various options for improving learning outcomes and instruction in Kenyan 
schools, using a randomized controlled design. The design is essential to this contribution, as 
many pilot programs in the sector do not test the impacts of quality-improvement methods at a 
medium scale and with enough rigor to identify a causal impact. In this paper, we focus on early- 
grade literacy outcomes — reading fluency and comprehension — after one year of implementation 
for pupils in grades 1 and 2. 

Setting: 

The PRIMR study was conducted in 502 government and nonformal slum schools in peri-urban 
and rural areas of Nairobi, Nakuru, and Kiambu counties in Kenya. Peri-urban regions are on the 
outskirts of urban areas — near enough that residents can commute to towns and cities via local 
transport, but still possessing many rural characteristics, such as agriculture being the 
predominant economic activity (Mandere, Ness, & Anderberg, 2010). In Nairobi, the largest city 
and capital of Kenya, more than half of the population lives in nonformal settlements, sometimes 
referred to as slums (United Nations Human Settlements Programme [UN-Habitat], 2013). Many 
of these settlements have no running water, access to electricity, or basic sanitation facilities. 
Low-cost, private nonformal schools are common alternatives to public government schools, 
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particularly at the primary level. The nonformal schools participating in PRIMR generally are 
characterized by low tuition rates (less than US$10 per month), substandard infrastructure 
(predominantly tin roofs and unfinished floors and walls), high student and teacher turnover, and 
lack of trained principals and teachers. 

Population / Participants / Subjects: 

At baseline, we sampled 2335 students in 117 of the 502 schools participating in the PRIMR 
initiative. At follow-up we sampled 2312 students in 117 schools. The students were in first and 
second grades, and were learning to read in both English and Kiswahili. 

Intervention / Program / Practice: 

The PRIMR interventions discussed here centered on improving teacher practices related to 
literacy acquisition, moving teachers beyond using whole-class oral repetition as their primary 
pedagogical approach toward research-supported strategies to improve bilingual literacy 
acquisition. The literacy arm of the program included 150 structured lessons in both Kiswahili 
and English. Teachers received modest instructional aids, including pocket charts and flashcards 
with letters on them, and students received low-cost student books that aligned with the scripted 
lessons. In PRIMR classrooms, the student-to-textbook ratio was 1:1 rather than the standard 3:1 
mandated by the national textbook policy and found in PRIMR schools at the baseline (Piper & 
Mugenda, 2012). The PRIMR books focused on letters, phonological awareness, and decoding 
skills — the building blocks of reading. They also exposed students to controlled-text stories 
relevant to their local context, as well as stories for teachers to read aloud, with a heavy emphasis 
on comprehension strategies. Together, the PRIMR lessons and materials were intended to move 
children who had not attended preschool and had little exposure to the alphabet from basic letter 
knowledge to full fluency and comprehension within one school year. 

A significant amount of PRIMR’s time and technical expertise was spent on teacher professional 
development. Each participating teacher and head teacher received 10 days of training during the 
first year of implementation. The training provided brief substantive overviews of reading topics, 
then allowed ample time for teachers to practice with the scripted lesson plans and activities. 
TAC tutors and instructional coaches, responsible for supporting teachers in clusters of schools, 
received 15 days of training to ensure that they would be capable of guiding teachers as they 
implemented the program. Given their critical role in shaping instructional improvement 
nationally in Kenya, these trainers are seen as critical to the program’s ongoing success and 
scalability. In fact, it is the TAC tutors (for zones of formal schools) and instructional coaches 
(for clusters of nonformal schools) who provided the training for teachers and head teachers as 
well as follow-up support. This means that the PRIMR model can be implemented within the 
existing MOE structures and available financial resources. 

Research Design: 

PRIMR is a randomized control trial of several instructional interventions. Random assignment 
to treatment or control (delayed commencement of program) group was conducted at the zone 
level — groups of 15 geographically proximate schools. For the purposes of this analysis of the 
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impact of the core PRIMR program, the baseline dataset included 2335 pupils in 117 schools. 
Schools were randomly selected from the cluster and zone of schools to which they belonged, 
and the sample of schools in the baseline included one half of the total number of schools in each 
zone. Enumerators selected the pupils using simple random sampling by having all of the 
students in each grade line up and then randomly selecting five boys and five girls each from 
grades 1 and 2, using a sampling interval derived from the student population. At the one-year 
midterm data collection in October 2012, the dataset utilized to measure program impact 
included 2312 students in 117 schools. 

Data Collection and Analysis: 

The baseline data collection was completed in January 2012, at the beginning of the school year, 
and the year one (midpoint) assessment was completed in October 2012. Oral reading fluency 
and reading comprehension were assessed using a version of the Early Grade Reading 
Assessment, or EGRA (see Gove & Wetterberg, 2011), adapted for use in Kenya. The measures 
focused on in this paper were those for the timed, read-aloud stories and their associated reading 
comprehension questions, as discussed further in the following section. The assessments were 
conducted by Kenyan field staff who had worked with PRIMR lead implementer RTI 
International since 2007 on several studies using EGRA. These assessors received five days of 
training before assessments commenced for both the baseline and year one studies. Interrater 
reliability scores were above 95% in both languages at both data collection rounds. 

The January 2012 baseline was designed to test whether there were statistically significant 
differences in the outcome variables between treatment groups. Although PRIMR randomly 
selected assigned schools to treatment groups, our analyses showed that treatment schools 
outperformed control schools by 2.7 WCPM in English reading (p-value .04), 1.7 WCPM in 
Kiswahili reading (p-value .04), 2.7% in English comprehension (p-value .08), and 5.4% in 
Kiswahili comprehension (p-value .07). Given the small but statistically significant differences 
in outcomes observed between the groups at the January 2012 baseline, we decided to use a 
difference-in-differences (DID) model to identify the effects of the intervention. We fit the DID 
estimator using Ordinary Least Squares (OLS) regression models with covariates for the fluency 
and comprehension outcomes. DID models compare changes in a program’s outcome variables 
at two different assessment points for treatment and control groups by removing the secular trend 
(the change in outcome for the control groups over time). This allows the analysis to separate 
program impact from changes in the population not due to program impact (Mumane & Willett, 
2011). DID models depend on the assumption that the trends in the covariates are the same in 
treatment and control groups (Mumane & Willett, 2011; Mumane, Willett, & Cardenas, 2006). 
In the case of PRIMR, this assumption will be tested at the final assessment in October 2013 
(Piper, 2009). 

The DID model was fit to a dataset that contained four groups of students, differentiated by 
whether they attended schools randomly assigned to treatment or control groups and by 
assessment round — January or October 2012. To answer the research questions in this paper, we 
fit the following statistical model: 

IMPACTij = p 0 + PiPRIMRij + /3 2 MIDTERM i j + p 3 PRIMR * MIDy + yX + (e y + p ; ) 
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for student i in school j, where s is an individual residual and n is the school-level residual, and 
the vector of covariates X , with associated regression parameters y, represents the impact of the 
control predictors. Parameter /? 0 is the intercept, /? x represents the main effect of PRIMR, and /? 2 
is the main effect of being in the midterm data set as opposed to the baseline. Parameter /? 3 
represents the impact of the interaction between PRIMR and MIDTERM , and is the DID 
parameter that is of principal interest, as the measure of PRIMR impact (Piper, 2009). Using the 
svy commands in Stata, we fit a statistical model in order to account for the nested nature of 
schools and students, and use standard errors that accounted for that nesting. 

Findings / Results: 

The analysis has shown that the first year of PRIMR had a positive impact on the three outcomes 
of interest — oral reading fluency, the percentage of pupils who read at the MOE’s benchmark, 
and reading comprehension — although not on all combinations of language, grade, and school 
type. Using the DID estimator, we found statistically significant impacts on oral reading fluency 
in grade 1 formal and nonformal schools and grade 2 nonformal schools for both English and 
Kiswahili (please see Table 1 in Appendix B). The relationship between PRIMR and oral reading 
fluency was nonsignificant in grade 2 formal schools, however. The magnitude of the impact was 
largest for grade 1 students in nonformal schools and was larger in English than in Kiswahili. In 
earlier research, we found that Kiswahili words were significantly longer than English words 
(Piper, 2010), providing one possible explanation for the disparity between English and 
Kiswahili fluency impacts. As a result of the improved performance, a larger percentage of 
children met the Ministry of Education’s fluency benchmarks (please see Figure 1 in Appendix 
B). 

Conclusions: 

The analysis has shown that the first year of PRIMR had a positive impact on the three outcomes 
of interest — oral reading fluency, the percentage of pupils who read at the MOE’s benchmark, 
and reading comprehension — although not on all combinations of language, grade, and school 
type. These findings suggest that programs like PRIMR could help Kenya reach its literacy 
goals. However, vast differences in educational participation and literacy have been observed 
across Kenya’s regions and levels of urbanicity (Mugo et al., 2011; Wasanga et al., 2010). 
Therefore, we must note that this intervention and the findings to date reflect a higher percentage 
of peri-urban areas than exists country-wide. The effects of PRIMR may vary by location. 
Analysis of the impact of the DFID-funded extension of PRIMR into two additional counties will 
allow for deeper examination of this issue. 

The meaningful first-year impacts of PRIMR on student achievement have implications for 
teacher professional development in Kenya. PRIMR’ s research has shown that teachers can be 
sensitive to in-service teacher professional development (ITPD) if that ITPD is closely linked to 
the books and lesson plans used in schools, and if teachers are observed and supported frequently 
(Mejia & Piper, in progress). PRIMR’s decision to invest heavily in classroom observational 
support (as compared to other programs focused on improving literacy outcomes) is important to 
note. Future research should examine how enhanced teacher support can change classroom 
behaviors. 
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Appendix B. Tables and Figures 


Table 1. Differences-in-differences estimates of PRIMR treatment effects on outcome 
measures, by grade and school type 


Outcome measure 

Language 

Metric 

Grade 1 

Formal Nonformal 

Grade 2 

Formal Nonformal 

Oral reading fluency 

English 

WCPM 

8.74** 

14.05*** 

2.62 

16.21*** 

Kiswahili 

WCPM 

3.31* 

10.97*** 

0.77 

12.88*** 

Proportion of readers at 

English 

% 

4.78- 

10.16*** 

12.09* 

21.07** 

the benchmark 

Kiswahili 

% 

7.21** 

15.49*** 

8.25* 

17.34** 

Reading 

English 

% 

3.07- 

3.15- 

-4.87 

9.06* 

comprehension 
(percentage correct) 

Kiswahili 

% 

5.81* 

12.00*** 

1.56 

9.64* 


WCPM = words correct per minute 
~p < .10, *p < .05, **p < .01, ***p < .001 


Figure 1. Fluent readers 

Kiswahili 


English 


17.6 


□ Baseline □ Normal Gain □ PRIMR Effect 

50 — 

45 
40 
35 
30 
25 
20 
15 
10 
5 
0 


6.6 


15.2 


8.5 

03 - 




7.6 






7.1 



5.4 



23.6 


6.8 


Formal |Nonformal! Formal |Nonformal 
Grade 1 Grade 2 


□ Baseline □Normal Gain □ PRIMR Effect 

60 -r— 


= 

£ 

c 

u 

22 

£ 


cx 

- 

C 

V 



SREE Spring 2014 Conference Abstract Template 


A-2 




